Wireless augmented reality communication system

ABSTRACT

A portable unit is for video communication to select a user name in a user name network. A transceiver wirelessly accesses a communication network through a wireless connection to a general purpose node coupled to the communication network. A user interface can receive user input to log on to a user name network through the communication network. The user name network has a plurality of user names, at least one of the plurality of user names is associated with a remote portable unit, logged on to the user name network and available for video communication.

GOVERNMENT LICENSE RIGHTS

The U.S. Government has certain rights in this invention pursuant toNAS7-1407 awarded by NASA.

CROSS-REFERENCE TO RELATED APPLICATIONS

Any and all applications for which a foreign or domestic priority claimis identified in the Application Data Sheet as filed with the presentapplication are hereby incorporated by reference under 37 CFR 1.57.

FIELD OF THE INVENTION

The invention, in general, relates to a wireless augmented realitysystem (WARS), and more particularly, to a WARS that leveragescommunications and multimedia information processing microelectronics,along with displays, imaging sensors, biosensors, and voice recognitionto provide hands-free, tetherless, real-time access and display ofnetwork resources, including video, audio and data.

DESCRIPTION OF RELATED INFORMATION

Online instruction manuals are becoming more prevalent in the industrialand everyday environment. These electronic technical manuals (ETM) maybe interactive. Just as with printed manuals, ETMs may become verydifficult to use and maintain in these environments where elements of anenvironment, such as dust, chemical or general harshness may bedetrimental to the electronics and storage devices used to display andoperate the ETM. Further, it is not always possible for a worker whorequires access to an ETM to stop work to consult ETM.

These problems are multiplied in extraterrestrial environments such as aspace shuttle or a space station. During intra and extra vehicularactivities, it may be virtually impossible to access a traditionalkeyboard and computer display to access an ETM. For example, during asatellite repair mission, it would not be practical for an astronaut ina bulky extravehicular space suit to type commands on a keyboard to viewa display in the extreme environment of outer space where the sun glaremay make viewing impossible.

Hands-free portable computers have been implemented in an attempt tosolve some of these problems. For example, U.S. Pat. Nos. 5,305,244 and5,844,824 describe systems in which a head-up display and voicerecognition is implemented in a portable computer for displaying ETM.However, these systems, being a single user-to-computer paradigm, do notallow multiple-user access to multiple computers, multimedia devices ornodes on a network for accessing arbitrarily-selected data channels.Further, these previously-described systems are self contained and theirdata storage needs to be updated periodically to be sure that the latestdata is displayed. Further, these systems do not allow two-waycommunication over local and wide area networks to other multi-mediausers and devices, and do not provide real-time biomedical informationabout the physical condition of the user.

There is thus a need for a wireless, wearable communications systemallowing two-way voice, video and data communication between local usersand to remote users and devices over network-nodes, along withtetherless real-time monitoring of the local user's physical condition.

SUMMARY

The needs of the prior art are met by a portable unit, methods ofsoftware for video communication to select a user name in a user namenetwork.

In one embodiment, a transceiver wirelessly accesses a communicationnetwork through a wireless connection to a general purpose node coupledto the communication network. A user interface can receive user input tolog on to a user name network through the communication network. Theuser name network has a plurality of user names, at least one of theplurality of user names being associated with a remote portable unit,logged on to the user name network and available for videocommunication. In some embodiments, the user interface comprises atouchpad configured to receive user inputs. A display on the portableunit displays one or more of the plurality of user names.

In an embodiment, the user interface further receives a selection of auser name from the plurality of user names. The display displays videocommunication received by the portable unit from the remote portableunit. The video communication is associated with the selected user name.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatic illustration of the components of the system ofthe present invention;

FIG. 2 is block diagram illustrating communications components used bythe personal access unit and general purpose node of the system of FIG.1; and

FIG. 3 is a flowchart illustrating a method performed using the systemof FIG. 1.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

With reference to FIG. 1, a diagram illustrating components of thesystem of the present invention is shown. The system may comprise smallpager-like devices called portable access units 100. The portable accessunits 100 are accessorizable for different “multimedia” interfaces fordisplay, camera, audio and sensor operation. Another embodiment of theportable access unit 100 a comprises a wearable headset and microphoneassembly 102 a.

The portable access units 100-100 a interface directly through wirelesslink with a network through a general purpose node 150. The generalpurpose node 150 allows wireless-to-wire communication with a localnetwork 170. The local area network 170 may be electrically connected toa wide area network or Internet 172 in order to connect to remote localarea networks 174. Alternatively, the general purpose node 150 may bedirectly connected to the wide area network 172. The general purposenode 150 may thus act as a router for routing video, display, audio andcontrol data packets between the portable access units 100 and other, orremote, portable access units 100 or remote media devices 125, 180, etcconnected to the networks 170-174. The connection with a network 170-174may occur directly in electrical connection with one of the networks170-174, or in wireless communication through a remote general purposenode 150 a that is electrically connected to the network. The portableaccess units 100 may provide communication to and from remote mediadevices comprising computers 180-182 running specialized client softwareor certain commercial multimedia Internet software products such asvideo conferencing products that adhere to the industry standard H.323for multimedia data transfer.

Each portable access unit 100-100 a may dynamically associate with theclosest general purpose node 150-150 a when it is logged on to thenetwork 170-174 or is connected thereto. Each general purpose node150-150 a records the associations and registers each portable accessunit 100-100 a on a list of connections associated with the particulargeneral purpose node 150-150 a. The list of connections is stored in arandom access memory device included in the general purpose node 150-150a. When a portable access unit 100 is logged off or disconnected fromthe network 170-174, it is disassociated from the particular generalpurpose node 150-150 a that it was associated with, and thus, is removedfrom the list of connections.

As shown on an example selection list screen 190 that may be presentedon a display 102 or headset 102 a on any of the portable access units100-100 a, the user can set up a video, audio, or data link with anyother portable access unit 100-100 a or remote media device 125, 180,etc, that is logged onto a network 170-174. The headset 102 a maycomprise a heads-up display (120 in FIG. 2) inside a headset embodying atransparent color LCD device. Using control keys or voice commands, auser of the portable access unit 100-100 a may select a local or remoteportable access unit 100-100 a on a selection list 190 of other portableaccess units 100-100 a or media devices 125, 180. The selection list 190comprises a combination of the lists of connections stored on all of thegeneral purpose nodes 150-150 a. Users may further access a nameserverlocated on the access node 150 for locating remote unfamiliar portableaccess units 100-100 a or remote media devices.

By selecting entries from the selection list 190, users may communicatewith portable access units 100-100 a or various media devices such ascameras 125, internet phones 104, one or more computers 180-182 locatedthroughout the networks 170-174. A user may further select from the list190 user names representing users of other portable access units 100that are logged in and associated with remote general purpose nodes 150a connected to the networks 170-174.

With reference to FIG. 2, the components of the access node 150 and thewearable headset embodiment of the portable access unit 100 a is shown.Elements for both the general purpose access node and portable accessunit 100 a include a communications device 202. Data processingfunctions are implemented in the form of an audio/video coder/decoder(codec) pair 200, one codec 200 comprising part of the portable accessunit 100 a and the other codec 200 being part of another portable accessnode 100 a or remote media device for which it is desired to exchangesignals. At a portable access node, the codec 200 controls a digitaldata stream which is fed to the communications device 202, which isimplemented as an RF modem transceiver pair with an equivalentcommunications device 202 located in the general purpose access node.The codecs 200 serve as the interfaces to the external elements(including possibly the user display 102 a and the sensor 104) on bothsides of the communication continuum comprising the communicationsdevice 202 of the general purpose node 150, an internal networkinterface protocol circuit 152, the external networks 170-174 and theelectrical connection or general purpose access node connection to thedesired remote portable access node or media device. The internalnetwork interface protocol circuit 152 may comprise an Ethernet chip,memory and a network protocol chip. With this architecture, the systemaddresses the issues of multiple-access and data channel quality,through the implementation of the communications device 202. Multipleimplementations of the communication device 202 in the general purposenode 150 allows for multiple simultaneous communication links with aplurality of portable access units 100-100 a for the general purposenode 150.

With the base functionality of the communications device 202 and codecsubsystem 200, the architecture provides flexibility in utilization ofdifferent external components such as different headset 102 aconfigurations, sensor 104 packages, and network interface 152capabilities.

The communication device 202 is designed to operate in a high multipathspace station or terrestrial indoor environment while being able tosupport multiple users at high, multimedia-type bandwidths. Thus thecommunications device's 202 physical (PHY) and media access (MAC) layersin combination support multiple access, dynamic network association,channel error rates of broadcast video quality (1.times.10e-6), and datarates up to broadcast-quality video bandwidths (on the order of 768 kbpsper user (one-way)). Modulation to achieve this performance will bedifferential phase-shift keying, of binary or higher order (quadratureor high-order quadrature amplitude modulation); the order chosenreflects the necessary user data volume to be supported within fixed,FCC-specified bandwidth allocations. Orthogonal frequency divisionmultiplexing, code division multiple access, and frequency hopping/timedivision multiple access may be used for achieving multiple access.Spread spectrum, channel equalization, antenna diversity andretransmission techniques may also be used for improving the reliabilityof the communications link. Through a combination of these technologies,two-way multimedia channel throughputs can be achieved for each ofmultiple simultaneous users. A variety of RF frequencies may be used,but the determining factor in frequency band selection is theavailability in the band of a relatively large amount of spectrum in thespace station or FCC terrestrial allocations, allowing the transmissionof compressed video. Ranges in the 2.5 to 5.7 band range are preferabledue to the FCC bandwidth available, the compactness of RF elementsrequired at these frequencies, and the potentially low amount ofinterference that will be sustained. The RF front end of both theportable access unit 100-100 a and general purpose node 150-150 a may beinterchangeable with different frequency front ends for system use indifferent frequency bands.

Low-rate, single user implementations of the communications system maybe effected through adapted commercial wireless-LAN type productsfollowing the FCC 802.11 standard such as a frequency-hopping 2.4 GHzwireless LAN transceiver by Waveaccess, Inc of Wellesley, Mass., ordirect-sequence spread-spectrum 2.4 GHz wireless LAN chipset by HarrisPrism of Melbourne, Fla. These radio implementations, as with commercialimplementations of the industry-proposed Bluetooth and HomeRF standards,will be limited in user access and overall throughput, however, andtherefore unsuitable to real-time video teleconferencing for multipleusers. The preferred embodiment for full capability is to implement thecommunications devices' physical and media access control layers incustom ASIC circuits allowing for support of all system capabilitieswithin microelectronics architecture for small size and low power draw,providing pager-type form factor of wearable personal access units100-100 a.

The communications device 202 comprises a buffer memory and a radiofrequency front end. Data modulation/demodulation circuits and errordetection/correction protocol circuits are further included. Variouscombinations of these circuits may be obtained from Proxim of Sunnyvale,Calif., Harris of Melbourne, Fla. and Stanford Telecom of Stanford,Calif. Alternatively, all of the various circuitry may be implementedwith an application specific integrated circuit (ASIC), or a combinationof an ASIC and discrete elements for size and weight efficiency.

Three classes of headsets 102 a may be used: hi-resolution militarysystems which are CRT based and may be provided by Honeywell ofMorristown, N.J., or Hughes Network Systems of San Diego, Calif.; mediumresolution industrial systems which are CRT or LED based scanners andmay be provided by Intervision of Santa Clara, Calif.; or low to mediumresolution entertainment systems which are color viewfinder LCD basedsystems that may be supplied by Virtual Vision of Redmond, Wash. (theV-CAP and E-GLASS), Sony Europe of Hampshire, United Kingdom (GLASSTRONVISOR) or Olympus of San Jose, Calif. Typical headset display 120specifications for the portable access unit 100 a include the following:

RESOLUTION: Comparable at least to VGA (640×480) or better to 1280×1024w/off-the-shelf display & I/O configuration

DISPLAY: >10 FL/day, Display Bright.Ratio: >2, Brightness range: 2OOM_(max)

FOV: 40-60 deg, Gray scale: >12

EyeRelief: 20-26 mm TSP, 14/10 mm (on/off-axis) exit pupil

Unif: 2:1 across ⅔ FOV, GLARE: <2.5% image content, PixelContrast: 25

FOCUS: Hands off, Obs: % look-around, Diopter range: .+−.2,

Mag: 1±p5%, Cont: >95%, motion sensor 10° cone, Inter. Eye. adj: 52-72mm

Image Enhan & IFF: Weaponsight, motion sensor and computer interface

The audio/video codec 200 in a portable access unit 100-100 a or otherclient device is based around a single chip, standards-based codec thataccepts analog or digital audio and video (i.e. NTSC or VGA) compressesthis input, and multiplexes the compressed data with an external datastream. The preferred industry standards are: ITU H.263 based video, ITUG.722 based audio, and ITU H.221 based multiplexing. The audio videocodec 200 in the portable access unit 100-100 a can establish a linkwith a similar audio/video codec 200 associated with another portableaccess unit 100-100 a or a remote media device 104, 125, 180 or 182. Thesignals from the codec 200 in the portable access unit 100 a outputs thereceived and decompressed remote signals from the device for which thelink was established. The interface between the codec 200 andcommunication device 202 as well as between the communication devices202 of the general purpose node 150-150 a and portable access unit100-100 a operate two-way with a high bandwidth suitable fortransmitting video. Of this bandwidth, the audio portion utilizes up to64 kbps and the data from the sensor 104 utilizes the required amountfor the type of sensor 104, with the remainder allocated to compressedvideo. The quality of the video at these data rates in excess of 128kbps is at least equivalent to video teleconferencing quality video.

The audio/video codec 200 portion of the portable access unit 100-100 amay further comprise video input and output ports, audio input andoutput ports, data input and output ports, and a the above-mentionedmultimedia processor chip for packaging signals for data compression anddecompression for transmission. Exemplary multimedia processors includethe VCPEX chip by 8.times.8, Inc. of Santa Clara, Calif. or digitalsignal processing chips by Texas Instruments and others. The audio/videocodec 200 further comprises a field processor gate array, electricallyprogrammable read-only memory and random access memory for processingand packaging signals for compression and decompression.

The sensor 104 may comprise a commercially available pulse oximetersensor or other type of bio-sensor. A pulse-oximeter sensor allows themeasurement of pulse rate and oxygen saturation of the blood. Data fromthe sensor 104 is transmitted to the general purpose node 150-150 a, andtransmitted to any remote media device connected to any of the networks170-172. The sensor 104 may comprise an “on body” wireless humanperformance and fatigue monitoring system that communicates with abelt-mounted transceiver/control module. The remote media device maycomprise a processor 180-182 for display or logging of the real-timesensor signals.

The headset 102 a comprises a heads-up display 120 comprising atransparent color LCD device for video signals received and processed bythe codec 200. The headset 102 a may further comprise, or have attachedthereto, an integrated microphone 122 for receiving voice commands fromthe user of the portable access unit 100 a or for communicating voicesignals to a remote portable access unit 100 or remote media device. Theheadset may further comprise a speaker 124 or earpiece unit forpresenting audio signals to the user. The portable access unit 100 a mayfurther comprise a digital camera 106 that may either be attached on theuser's person, or to the headset 102 a for providing video signals toother portable access units 100-100 a or media devices.

With reference to FIG. 3, a flow diagram illustrating the methodperformed by the system of FIG. 1 is shown. A user puts on the headset102 a, portable access unit 100 a, step 400. The user may log into thelocal general purpose node 150 wherein the portable access unitassociates with the general purpose node 150 such that the user is addedto a connection list stored in a random access memory device residing inthe general purpose node 150, step 401. Data is provided from thegeneral purpose node 150 to the portable access unit through thecommunication devices 202, step 402. The user is presented with aselection list 190 of portable access units 100-100 a and media deviceslogged onto the system on the display 120, step 404. The user selectsone of the entries from the selection list, step 406. The selection istransmitted to the general purpose node 150, step 408. The generalpurpose node 150 sets up a connection over the networks 170-174 forchanneling data between the portable access unit 100 a and the selectednetwork device, step 410. The selected network device may comprise theprocessor 180 or other network client 182 for running a softwareapplication, a camera 125 for providing remote viewing operations to theuser on the display 120, the Internet phone 104 for providing voicecommunications with the a remote user, or another portable access unit100-100 a over a remote general purpose node 150 a. By providing controlcommands to the microphone 122 or other input system, such as a keyboardor handheld mouse, the user may conduct operations by transmittingcommands between the portable access unit 100 a and the general purposenode 150 which routs the control commands to the device that the userselected, step 412.

It will thus be seen that changes may be made in carrying out the abovesystem and method and in the construction set forth without departingfrom the spirit and scope of the invention, it is intended that any andall matter contained in the above description and shown in theaccompanying drawings shall be interpreted as illustrative and not in alimiting sense.

What is claimed is:
 1. A system for communication between remotedevices, the system comprising: a network device connected to acommunication network; and a memory associated with the network deviceand configured to maintain and update a connection list of wirelesscommunication devices that are currently connected to the communicationnetwork and ready for two-way communication with another networkeddevice such that the connection list is updated to add a new device whenthe new device is logged on to the communication network and that theconnection list is updated to remove an existing device when theexisting device is logged off from the communication network, wherein,in response to logon to the communication network by a first wirelessdevice associated with a first user name, the network device isconfigured to communicate with the first wireless device to cause todisplay, on an integrated display of the first wireless device, usernames to select from for two-way communication via the communicationnetwork and further to cause the user names to be updated based on theconnection list such that when a second wireless device corresponding toa second one of the displayed user names is logged off from thecommunication network, the second user name is no longer displayed onthe integrated display of the first wireless device, wherein, inresponse to selection of a third user name from the displayed usernames, the network device is configured to coordinate establishing awireless communication channel for video communication between the firstwireless device and a third wireless device corresponding to the thirduser name.
 2. The system of claim 1, wherein, in response to connectionto the communication network by the first wireless device comprising anintegrated camera, the integrated display and an integrated touch inputdevice, the network device is configured to cause the memory to add thefirst wireless device to the connection list.
 3. The system of claim 1,wherein, in response to disconnection from the communication network bythe first wireless device comprising an integrated camera, theintegrated display and an integrated touch input device, the networkdevice is configured to cause the memory to remove the first wirelessdevice from the connection list.
 4. The system of claim 1, wherein thefirst wireless device comprises an integrated microphone configured tocapture audio signals to be transmitted to the third wireless device,and the third wireless device comprises an integrated speaker configuredto receive audio signals transmitted from the first wireless device,such that upon the establishment of the wireless communication channel,the first wireless device is configured to provide real-time access tothe audio signals captured by the integrated microphone of the firstwireless device to the third user via the integrated speaker of thethird wireless device.
 5. The system of claim 1, wherein the firstwireless device comprises an integrated speaker configured to presentaudio signals received from the third wireless device, and the thirdwireless device comprises an integrated microphone configured to captureaudio signals to be transmitted to the first wireless device, such thatupon the establishment of the wireless communication channel, the firstwireless device is configured to provide real-time access to the audiosignals captured by the integrated microphone of the third wirelessdevice to the first user via the integrated speaker of the firstwireless device.
 6. The system of claim 1, wherein in response toselection of the third user name from the displayed user names, thefirst wireless device is configured to transmit the selection to thenetwork device.
 7. The system of claim 1, wherein the third wirelessdevice comprises a processor configured to execute a video communicationsoftware application configured to provide real-time access to videosignals captured by the third wireless device to the first wirelessdevice.
 8. The system of claim 1, wherein the third wireless devicecomprises an integrated camera configured to capture video signals to betransmitted to the first wireless device such that upon theestablishment of the wireless communication channel, the first wirelessdevice is configured to provide real-time access to the video signalscaptured by the integrated camera of the third wireless device to thefirst user via the integrated display of the first wireless device. 9.The system of claim 1, wherein the first wireless device comprises anintegrated camera configured to capture video signals to be transmittedto the third wireless device, and the third wireless device comprises anintegrated display configured to display video signals received from thefirst wireless device, such that upon the establishment of the wirelesscommunication channel, the first wireless device is configured toprovide real-time access to the video signals captured by the integratedcamera of the first wireless device to the third user via the integrateddisplay of the third wireless device.
 10. The system of claim 1, whereinthe first wireless device comprises an integrated input deviceconfigured to receive control commands such that upon the establishmentof the wireless communication channel, the first wireless device isconfigured to transmit the control commands received by the integratedinput device to the third wireless device to control at least oneoperation of the third wireless device based on the control commands.11. The system of claim 1, wherein the first wireless device comprises acodec configured to encode and decode video data transmitted between thefirst wireless device and the third wireless device.
 12. A system forcommunication between remote devices, the system comprising: a networkdevice connected to a communication network; and a memory associatedwith the network device and configured to maintain and update aconnection list of wireless communication devices that are currentlyconnected to the communication network and ready for two-waycommunication with another networked device such that the connectionlist is updated to add a new device when the new device is logged on tothe communication network and that the connection list is updated toremove an existing device when the existing device is logged off fromthe communication network, wherein, in response to logon to thecommunication network by a first wireless device associated with a firstuser name, the network device is configured to communicate with thefirst wireless device to cause to display, on an integrated display ofthe first wireless device, user names to select from for two-waycommunication via the communication network, wherein, in response toselection of a second user name from the displayed user names, thenetwork device is configured to coordinate establishing a wirelesscommunication channel for video communication between the first wirelessdevice and a second wireless device corresponding to the second username.
 13. The system of claim 12, wherein the first wireless devicecomprises an integrated microphone configured to capture audio signalsto be transmitted to the second wireless device, and the second wirelessdevice comprises an integrated speaker configured to receive audiosignals transmitted from the first wireless device, such that upon theestablishment of the wireless communication channel, the first wirelessdevice is configured to provide real-time access to the audio signalscaptured by the integrated microphone of the first wireless device tothe second user via the integrated speaker of the second wirelessdevice.
 14. The system of claim 12, wherein the first wireless devicecomprises an integrated speaker configured to present audio signalsreceived from the second wireless device, and the second wireless devicecomprises an integrated microphone configured to capture audio signalsto be transmitted to the first wireless device, such that upon theestablishment of the wireless communication channel, the first wirelessdevice is configured to provide real-time access to the audio signalscaptured by the integrated microphone of the second wireless device tothe first user via the integrated speaker of the first wireless device.15. The system of claim 12, wherein in response to selection of thesecond user name from the displayed user names, the first wirelessdevice is configured to transmit the selection to the network device.16. The system of claim 12, wherein the second wireless device comprisesa processor configured to execute a video communication softwareapplication configured to provide real-time access to video signalscaptured by the second wireless device to the first wireless device. 17.The system of claim 12, wherein the second wireless device comprises anintegrated camera configured to capture video signals to be transmittedto the first wireless device such that upon the establishment of thewireless communication channel, the first wireless device is configuredto provide real-time access to the video signals captured by theintegrated camera of the second wireless device to the first user viathe integrated display of the first wireless device.
 18. The system ofclaim 12, wherein the first wireless device comprises an integratedcamera configured to capture video signals to be transmitted to thesecond wireless device, and the second wireless device comprises anintegrated display configured to display video signals received from thefirst wireless device, such that upon the establishment of the wirelesscommunication channel, the first wireless device is configured toprovide real-time access to the video signals captured by the integratedcamera of the first wireless device to the second user via theintegrated display of the second wireless device.
 19. The system ofclaim 12, wherein the first wireless device comprises an integratedinput device configured to receive control commands such that upon theestablishment of the wireless communication channel, the first wirelessdevice is configured to transmit the control commands received by theintegrated input device to the second wireless device to control atleast one operation of the second wireless device based on the controlcommands.
 20. The system of claim 12, wherein the first wireless devicecomprises a codec configured to encode and decode video data transmittedbetween the first wireless device and the second wireless device.